Papers with generative modeling
Task-driven Layerwise Additive Activation Intervention (2025.naacl-short)
Copied to clipboard
| Challenge: | Existing approaches to task adaptation rely heavily on heuristic rules or prompt inputs. |
| Approach: | They propose a layer-wise additive activation intervention framework that steers the LMs’ generation process by identifying and manipulating the activations. |
| Outcome: | The proposed framework improves the accuracy of pretrained LMs and competing baselines on various datasets, demonstrating improvements in the accuracy and sample efficiency of the proposed framework. |
LanguageFlow: Advancing Diffusion Language Generation with Probabilistic Flows (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent work has demonstrated success in controlling sentence attributes and structure based on diffusion language models. |
| Approach: | They propose a language-rectified flow method that reformulates standard probabilistic flow models to learn ordinary differential equations to transport between the source and target distributions. |
| Outcome: | The proposed method outperforms baselines on three fine-grained control tasks and multiple high-quality text editing tasks. |
Compete to Complete: Co-opetition Adversarial Learning for Retrieval-Augmented Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to reduce hallucination in large language models lack a robust mechanism for generating a generative model. |
| Approach: | They propose a framework that formulates retriever–generator training in RAG as a minimax game. |
| Outcome: | The proposed framework improves retrieval-augmented generation performance on seven benchmark datasets. |
Latent Reasoning for Low-Resource Question Generation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to multihop question generation require extensive data collection and decomposition. |
| Approach: | They propose a generative approach that optimizes the two-phase model without question decomposition data. |
| Outcome: | The proposed approach outperforms baselines on HOTPOTQA, a benchmark multi-hop question answering dataset. |
Sequential Compositional Generalization in Multimodal Models (2024.naacl-long)
Copied to clipboard
| Challenge: | a growing number of multimodal models have a limited capacity for generalization . however, prior studies into compositionality have focused on visual grounding and downstream tasks like image captioning. |
| Approach: | They examine compositional generalization using egocentric kitchen activity videos . they find bi-modal and tri-modal models exhibit a clear edge over their text-only counterparts . |
| Outcome: | The proposed model outperforms text-only models in a multimodal setting. |
Language Modeling for Code-Switching: Evaluation, Integration of Monolingual Data, and Discriminative Training (D19-1)
Copied to clipboard
| Challenge: | Code-switching (CS) is a linguistic phenomenon defined as "the alternation of two languages within a single discourse, sentence or constituent." |
| Approach: | They propose an ASR-motivated evaluation setup which is decoupled from an ASL system and the choice of vocabulary . they propose a discriminative training approach which works better than generative language modeling . |
| Outcome: | The proposed evaluation setup is better than generative language modeling, the authors show . the proposed setup is decoupled from an ASR system and the choice of vocabulary . |
A Generative Framework for Personalized Sticker Retrieval (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing relevance-based generative retrieval methods lack personalization, leading to a mismatch between diverse user expectations and the retrieved results. |
| Approach: | They propose a representation learning model that learns discriminative user representations to encode user-specific sticker preferences. |
| Outcome: | The proposed framework outperforms state-of-the-art methods in generating relevant stickers for queries. |
Do Transformers Parse while Predicting the Masked Word? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that pre-trained language models encode linguistic structures like parse trees while being trained unsupervised. |
| Approach: | They propose to train pre-trained language models to encode linguistic structures like parse trees while unsupervised. |
| Outcome: | The proposed model performs optimally for masked language modeling loss on the English PCFG. |